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ABSTRACT 

A  system  under  development  for  automated  interpretation  of  oceanographic  satellite  images 
includes  a  Gulf  Stream  description  module  which  uses  a  neural  network,  which  produces 
coefficients  of  an  empirical  orthogonal  function  (EOF)  series  representation  of  the  Gulf 
Stream  directly  from  processed  satellite  imagery.  The  Gulf  Stream  module  consists  of  the 
EOF  software  and  the  neural  network,  with  input  from  an  innovative  edge  detector. 

The  Gulf  Stream  is  the  swiftest  and  most  energetic  curicnt  in  the  north  Atlantic,  and 
meanders  with  a  broad  spectrum  of  variability  on  several  spatial  and  temporal  scales. 
Satellite  observations  provide  a  means  to  observe  the  Gulf  Stream's  shape,  although  clouds 
in  IR  imagery  and  other  types  of  "noise"  complicate  interpretation.  The  Gulf  Stream  shape 
at  any  time  may  be  represented  as  a  series  of  complex  EOFs  (CEOFs),  i.  e.,  principal 

components,  which  can  be  truncated  after  a  relatively  small  number  of  terms  (10)  and  still 
describe  Gulf  Stream  shapes  well  (to  within  10  km).  These  modes  can  be  optimized  from 
initial  values  with  as  few  as  21  fixes  on  the  position  of  the  Gulf  Stream  axis,  using  least- 
squares  estimation.  The  CEOFs  interpolate  between  spatially  intermittent  observations  of 
portions  of  the  Gulf  Stream,  as  might  come  from  IR  imagery  with  partial  cloud  cover. 

The  study  described  here  tested  whether  a  credible  Gulf  Stream  can  be  produced  using  a 
neural  network  (simulated  in  software)  that  has  inputs  derived  from  imagery,  and  has  CEOF 
coefficients  as  outputs.  To  demonstrate  feasibility  we  considered  it  sufficient  to  define  the 
values  of  the  first  three  complex  mode  coefficients,  which  account  for  more  than  97%  of  the 
Gulf  Stream's  position  variance.  Two  data  sets  were  used  for  training  and  testing:  Gulf 
Stream  analyses  from  NOARL's  GEOSAT  Ocean  Applications  Program  (GOAP),  and  edge  images 
produced  from  IR  imagery  by  a  NOARL-developed  algorithm. 

In  the  first  part  of  the  study  the  input  consisted  of  132  latitude-longitude  pairs  that 
represent  one  Gulf  Stream.  The  learning  algorithm  is  back  propagation;  convergence  is 
achieved  with  100,000  iterations  through  the  training  set  with  the  learning  coefficient  set 
to  0.9  and  the  momentum  coefficient  set  to  0.6.  The  neural  network  has  one  hidden  layer 

with  10  nodes.  The  output  layer  has  6  nodes,  giving  the  real  and  imaginary  parts  of  each  of 
the  three  CEOF  mode  coefficients.  Training  using  the  GOAP  data  set  produces  outputs  that 

have  correlation  coefficients  with  the  correct  mode  coefficients  averaging  over  0.98. 

The  next  part  of  the  study  used  a  network  with  a  50  x  50  grid  (simulating  pixel  input)  of 
input  nodes,  one  hidden  layer  of  40  nodes,  and  6  output  nodes.  Again,  learning  was  by  back 
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propagation.  The  training  set  consisted  of  77  of  the  86  GOAP  data  sets,  with  9  randomly 

chosen  sets  held  back  as  test  data.  The  network  converged  after  150,000  passes  through  the 

test  data  set,  with  results  of  comparable  accuracy  to  those  obtained  with  the  first  network. 

Processing  to  produce  three  complex  mode  coefficients  from  50  x  50  griddeu  input  took  less 
than  1  second  on  a  20  MHz  PC. 

The  grid-input  network  applied  to  the  9  GOAP  test  cases  achieves  agreement  with  a 
correlation  coefficient  averaging  0.97.  The  results  are  moderately  good  for  the  noisy  edge 
imagery,  also.  The  coefficients  have  reasonable  values,  the  correlation  coefficient  between 
the  network  and  actual  coefficients  averages  0.69,  and  the  resulting  reconstructed  Gulf 
Stream  is  within  100  km  of  the  actual  Gulf  Stream  over  most  of  its  length.  This  is  notable 
because  the  resolution  of  the  50  *  SO  arid  is  59  jjm. 

The  studies  performed  so  far  demonstrate  feasibility.  The  next  phase  of  work  is  directed 

toward  making  the  system  operationally  useful. 

1.  INTRODUCTION 

This  work  is  part  of  an  effort  to  develop  and  demonstrate  techniques  for  automated 

interpretation  of  satellite  data  for  ocean  and  ice  applications.  Generation  of  oceanographic 
products  by  conventional  manual  means  requires  much  labor  and  very  high  skill  levels, 
both  of  which  are  difficult  to  realize  in  an  operational  environment.  The  satellite 

oceanography  image  understanding  problem  is  difficult  because  the  features  arc  naturally 
time-varying,  there  are  no  straightforward  mathematical  models  of  their  shapes,  and 
interference  by  cloud  cover  and  other  effects  is  a  frequent  problem.  To  address  these 
difficulties,  the  technical  issues  the  project  considers  are  image  segmentation  (simplifi¬ 

cation),  feature  labeling  (assignment  of  oceanographic  identity  to  image  features),  feature 
aggregation  (formation  of  Gulf  Stream  north  wall,  eddies,  etc.),  and  mesoscale  extrapolation 
(to  deal  with  data  gaps).  This  paper  describes  an  innovative  approach  to  the  feature 

labeling  and  feature  aggregation  portions  of  the  automated  Gulf  Stream  detection  problem. 

Satellite  imagery,  by  its  capability  for  synoptic  coverage  of  large  areas,  is  a  useful 

complement  to  traditional  in  situ  oceanographic  measurements.  Hawkins,  Phoebus,  and 

May1  discuss  the  assimilation  of  satellite  data  along  with  other  types  of  information  to 

produce  a  superior  description  of  oceanic  parameters.  During  the  past  few  decades,  new 

observations  have  begun  to  identify  a  phenomenon  that  appears  to  be  intimately  linked 
physically  to  the  ocean's  general  circulation,  and  even  to  dominate  the  circulation 
energetically  in  many  regions.  This  phenomenon  is  comprised  of  slow,  medium-sized 
fluctuations  in  the  circulation  itself.  This  "mesoscale"  variability  can  take  several  forms, 

such  as  meandering  of  the  Gulf  Stream,  large  ring  vortices  that  are  snapped  off  from  the 
currents  during  intense  meandering  events,  and  the  mid-ocean  eddies. 

The  sea  surface  is  teeming  with  thermal  structure  detectable  by  satellite  infrared  (1R) 

sensors.  Present  sensors  are  capable  of  1  km  horizontal  resolution  with  0.5°C  accuracy 

under  cloud-free  conditions.  The  Gulf  Stream  and  its  rings  contribute  to  the  surface 
thermal  expressions  observed  by  satellite.  Specifically,  the  Gulf  Stream  is  a  continuous 

feature  in  the  north  Atlantic  Ocean  whose  surface  thermal  expression  consists  of  sea 

suiface  temperature  gradients  between  the  longitudes  75° W  and  40° W.  Images  can  be 
produced  in  which  pixels  have  been  identified  that  are  associated  with  high  horizontal 
gradients  and  thus  are  candidates  for  the  Gulf  Stream  edge.  The  problem,  then,  is  to  discard 
some  of  these  high  gradient  edges  and  connect  the  rest  into  a  continuous  Gulf  Stream.  A 
mathematical  description  of  a  continuous  Gulf  Stream  with  realistic  meanders  has  been 
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developed.  The  innovative  step  described  in  this  paper  is  the  use  of  the  new  technology  of 

neural  networks  to  connect  the  gradients  of  IR  imagery  to  the  mathematical  description  of 

a  continuous  Gulf  Stream.  The  technique  has  the  potential  to  apply  the  great  speed  and 
quantitative  effectiveness  of  artificial  intelligence  technology  to  the  identification  of 

oceanographic  features  from  remotely  sensed  imagery.  This,  in  turn,  offers  the  benefit  of 
more  effective  use  of  the  U.  S.  Navy  staff  assigned  to  the  task  in  the  Fleet. 

The  emerging  capabilities  of  neural  networks  to  reduce  noisy  imagery  to  meaningful 
information  in  a  fast,  highly  parallel  fashion  presents  an  opportunity  to  solve  longstand¬ 
ing  problems  in  satellite  data  analysis.  A  neural  network  implemented  in  either  software  or 
hardware  greatly  decreases  the  need  for  a  human  operator  to  examine  the  IR  imagery, 
since  the  network  operates  directly  on  the  data  to  produce  probable  Gulf  Stream  points. 

This  computer  implementation  yields  another  immediate  bencfit--the  processing  begins  as 
soon  as  the  satellite  data  are  acquired  instead  of  waiting  for  a  human  expert  to  analyze  the 
data.  The  necessity  for  specialized  shore-based  display  equipment  and  interactive  software 
are  also  diminished  because  the  network  operates  on  the  digital  data  directly  as  it  is 
received  from  the  satellite  or  automatically  processed  by  existing  edge  detection  algorithms. 

In  addition  to  countering  the  deficiencies  outlined  above,  a  neural  network  capable  of 
analyzing  IR  imagery  opens  up  new  possibilities  in  the  U.  S.  Navy’s  use  of  satellite  data.  The 
capability  to  analyze  satellite  imagery  without  the  use  of  an  expert  and  extensive 
workstation  introduces  the  possibility  of  onboard  analysis  of  data.  This  enables  properly 

equipped  vessels  to  make  critical  decisions  based  on  the  latest  information  in  near-real¬ 
time  without  the  delays  of  a  system  dependent  on  shore-based  facilities. 

During  the  early  1990s  the  U.  S.  Navy  will  face  an  unprecedented  wealth  of  new  remotely 
sensed  oceanographic  data  available  from  both  U.  S.  (e.  g.,  TOPEX/Poseidon,  Spinsat)  and 

foreign  (European  ERS-1,  Japanese  ERS-1)  satellites.  Present-day  satellites  will  continue  to 
supply  oceanographic  data  into  the  forsecablc  future.  The  sheer  volume  of  data  from  these 
satellites  will  be  overwhelming  to  analysts,  and  anticipated  manpower  reductions  will 
dictate  a  need  for  greater  automation.  This  data  volume  increase  will  coincide  with  the 

deployment  of  the  third  version  of  the  Tactical  Environmental  Support  System  (TESS  (3)),  a 
capable  environmental  information  system  which  will  extend  satellite  data  processing 
capability  to  approximately  70  U.  S.  Navy  ships  and  shore  facilities.  Thus,  the  work 

described  in  this  paper  is  coming  along  at  the  right  lime. 

2.  BACKGROUND 

A  useful  numerical  description  of  the  Gulf  Stream  is  one  based  on  complex  empirical 

orthogonal  functions  (CEOFs),  the  eigenvectors  of  an  "expectation  matrix."  The  expectation 
matrix  is  obtained  by  averaging  a  set  of  matrices  each  representing  the  correlation  proper¬ 

ties  of  an  individual  vector,  in  our  case  a  vector  that  describes  Ihe  instantaneous  position  of 
the  Gulf  Stream.  This  formulation  and  its  advantages  for  describing  the  Gulf  Stream  arc 

described  in  detail  by  Carter. 2  To  summarize  that  reference,  a  looping,  contorted,  but 
continuous  Gulf  Stream  can  be  described  by  a  vector  of  complex  elements  consisting  of 
(latitude,  longitude)  pairs.  That  vector  can  be  approximated  well  by  a  relatively  small  set  of 
complex  cocfficients--when  each  coefficient  is  multiplied  by  a  corresponding  CEOF  (princi¬ 
pal  component,  eigenvector,  or  mode)  that  is  fixed  for  all  time.  The  resulting  terms  are  a  set 
of  vectors  which  are  added  together  to  reproduce  the  original  Gulf  Stream  vector.  Speaking 

geometrically,  the  CEOFs  constitute  a  set  of  basis  vectors  that  span  the  space  of  all  known 
Gulf  Stream  states  (i.  c.,  locations  and  shapes);  an  individual  Gulf  Stream  state  is  produced  by 
linear  combinations  of  the  basis  set.  Speaking  figuratively,  the  CEOFs  are  the  shapes  of  Gulf 
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Stream  patterns  and  the  coefficients  are  the  orientations  and  sizes  of  the  patterns.  The 
power  of  the  method  derives  from  the  fact  that  a  partial  sum,  involving  a  relatively  small 
number  of  CEOFs,  may  reproduce  the  Gulf  Stream's  shape  with  sufficient  accuracy. 

Mathematically,  the  Gulf  Stream  is  defined  by  a  complex  vector  W  whose  elements  are: 

Wj  =  xj  +  i  yT  (1) 

In  this  equation,  x  is  the  longitude  and  y  is  the  latitude  of  a  point  on  the  Gulf  Stream  axis. 
This  complex  formulation  allows  the  description  of  a  contorted  Gulf  Stream  which  may 
curve  back  on  itself,  while  a  description  in  terms  of  y  as  a  function  of  x,  or  vice  versa, 
would  have  to  deal  with  double-valued  functions.  The  expectation  matrix  A  is  defined  as  the 
matrix  with  elements: 


Aij  =  (wiW“),  (2) 

where  *  denotes  the  complex  conjugate  and  <  >  is  the  expectation  operator.  A,  which  is 
generated  from  an  ensemble  of  Gulf  Stream  axes,  is  Hermitian,  so  it  has  real  eigenvalues 

(but  complex  eigenvectors).  Given  the  eigenvectors  of  A,  and  one  particular  realization, 

W,  of  the  Gulf  Stream,  the  complex  mode  coefficients  associated  with  W  may  be  found  from: 

ck  =  e"  W,  (3) 

where  the  superscript  H  denotes  the  Hermitian  conjugate.  The  coefficients  allow  the 
realization  W  of  the  Gulf  Stream  to  be  reconstructed  using  the  expansion: 

N 

W  -  XCkEb  (4) 

k=4 

where  N  is  the  number  of  eigenvectors. 

The  eigenvectors  of  A  form  a  complete  set  of  orthogonal  basis  vectors.  If  the  eigenvectors 

are  sorted  according  to  the  size  of  the  corresponding  eigenvalues  (k  =  1  has  the  largest 
eigenvalue,  etc.),  then  it  is  easily  shown  that  the  first  eigenvector  is  the  best  fit  in  a  least- 
squares  sense  to  the  "cloud"  of  points  in  the  mathematical  space  in  which  each  Gulf  Stream 

(vector)  defines  one  point.  That  is,  the  sum  of  squares  of  the  total  distances  between  the 

points  and  the  "line"  are  a  minimum.  (This  is  analogous  to  what  Duda  and  Hart  call 

"eigenvector  line  fitting. ”3)  So,  the  first  eigenvector  is  aligned  with  the  direction  of 
maximum  scatter  (i.  c.,  variance)  of  the  points.  Likewise,  the  second  eigenvector  "explains" 

the  next  largest  amount  of  variance,  etc.  In  fact,  each  eigenvalue  is  proportional  to  the 

amount  of  variance  that  a  given  eigenvector  explains.4  An  immediate  consequence  of  this 

is  that,  in  practice,  Eq.  (4)  may  be  truncated  after  a  relatively  small  number  of  terms  and 

still  reconstruct  a  particular  Gulf  Stream  with  sufficient  accuracy.  Figure  1  illustrates  the 

reconstruction  of  a  convoluted  Gulf  Stream  using  the  first  10  of  132  eigenvectors. 

Many  readers  are  familiar  with  normal  mode  analysis  and  have  seen  the  above  formulation 

in  other  contexts.  Also,  the  reconstruction  algorithm  appears  to  fail  when  one  docs  not 

have  a  complete  description  of  a  Gulf  Stream  (i.  e.,  some  of  the  elements  of  W  arc  missing) 

because  of  cloud  cover  or  some  other  form  of  interference.  In  thi*  case,  NOARL’s  software 

employs  a  gradient-search  technique  to  perform  a  least-squares  fit  ot  the  CEOF  expansion  to 
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the  portions  of  the  Gulf  Stream  which  arc  observed.  The  coefficients  found  in  a  (recent) 
previous  analysis  are  suitable  initial  estimates. ^ 

This  latter  step  is,  itself,  an  innovative  approach  to  providing  a  mathematical  description  of 

a  realistic,  continuous  Gulf  Stream  from  incomplete  observational  data.  A  further 
development  involves  the  application  of  neural  network  technology.  Neural  networks  have 
been  emerging  as  a  new,  viable  technology  for  image  analysis  and  pattern  recognition. 
There  are  several  characteristics  of  neural  networks  that  make  them  effective  for 
analyzing  large  amounts  of  data  rapidly  and  resistant  to  noise  in  the  data.  Neural  networks 
are  composed  of  many  simple  computational  units  that  work  in  unison  to  process 
information.  The  network  is  not  programmed  in  the  traditional  sense;  rather,  the  weights 
that  connect  the  neurons  are  "learned"  by  the  network  through  repeated  applications  of 
input  data  for  which  the  desired  result  is  known.  Thus,  a  network  can  be  presented  with  an 
IR  image  or  other  input  data,  and  it  can  process  that  data  based  on  the  information 

presented  during  training.  The  network  is  tolerant  of  noise  since  it  has  learned  the  salient 
features  of  an  image  during  training,  and  the  noise  is  in  some  sense  averaged  out. 

NOARL  has  developed  an  edge-detection  algorithm,  the  cluster-shade  edge  detector  (CSED),  to 

compute  the  position  of  sea-surface  temperature  (SST)  gradient  fronts  in  IR  imagery.6  That 
algorithm  has  several  advantages  for  this  application,  compared  to  other,  well-known  edge 

detectors,  such  as  the  Sobcl  algorithm.7  The  immediate  need  is  to  identify  the  Gulf  Stream 

among  the  fronts.  Alternatively,  we  may  say  that  the  Gulf  Stream  is  the  pattern  to  be 
recognized  in  a  CSED  image,  and  othet  fronts  constitute  noise  in  that  edge  image.  This  paper 
describes  an  exploration  of  the  capacity  of  neural  networks  to  recognize  the  Dull  Stream 

pattern,  as  represented  by  CEOF  coefficients,  within  the  noise  of  an  SST  image  processed  by 
the  CSED  algorithm. 

3.  ISSUES  IN  NEURAL  NETWORK  PRODUCTION  OF  CEOF  COEFFICIENTS 

The  basic  question  to  be  answered  was  whether  a  neural  network  can  produce  meaningful 
coefficients  for  CEOFs  whose  structures  w-crc  not  available  to  t lie  network.  That  is.  are  the 

patterns  represented  by  coefficients  of  CEOFs  appropriate  for  neural  network  technology? 
in  answer  this  question,  it  was  necessary  to  consider  several  subquestions,  or  issues: 

1  What  constitutes  an  appropriate  training  set  for  the  network? 

2.  What  is  the  criterion  for  a  meaningful  CEOF  coefficient? 

3.  What,  if  arty,  neural  network  training  procedure  is  capable  of  producing  mean¬ 
ingful  coefficients  for  CEOFs  from  the  chosen  training  set? 

d  What  is  an  appiopriate  configuration  of  nodes  and  layers  to  produce  meaningful 

Ci  OF  cocfTicwnts? 

\  How  sensitive  to  the  configuration  of  hidden  layers  and  nodes  is  the  pioduciion  of 
meaningful  (’EOF  coefficients.’ 

CEOFs  computed  by  Molinelli  and  Flanigan'  were  expressed  as  vectors  with  elements  every 
ten  nautical  miles  downstream.  Operational  use  would  provide  latitudes  and  longitudes  of 
possible  Gulf  Stream  locations  derived  from  SST  edge  imagery  that  could  not  be  associated  a 
priori  with  distance  downstream.  Consideration  of  this  fact  led  to  the  next  issue: 

6.  (an  input  nodes  representing  positions  on  a  latitude-longitude  grid  produce 
meaningful  CEOF  coefficients? 

It  was  deemed  necessary  to  resolve  these  issues  before  getting  to  details  ot  noise  discrimi¬ 
nation,  conditioning  of  input  data,  and  increase  in  resolution  and  accuracy  of  the  neural 
network  output. 
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4.  APPROACH 


A  simple  approach  was  employed  in  investigating  the  above  questions.  That  approach  was 
to  use  a  series  of  actual  Gulf  Stream  representations  produced  between  January  1986  and 
June  1987  to  develop  a  neural  network  and  test  its  capabilities.  The  Gulf  Stream  represen¬ 
tations  came  from  the  "mesoscale  products"  of  the  GEOSAT  Ocean  Applications  Program 
(GOAP),8.  9  and  were  derived  by  a  human  analyst  using  not  only  IR  SST  imagery  but  also 
GEOSAT  altimetric  profiles.  There  arc  86  Gulf  Streams  represented  by  latitude-longitude 
pairs  at  inflection  points  stored  in  computer  files.  This  set  of  Gulf  Streams  constitutes  the 
first  candidate  training  set  for  the  present  study. 

We  used  the  CEOF  software  developed  by  Molinelli  and  Flanigan,5  and  started  by  deriving 
new  CEOFs  for  the  training  data  set.  To  demonstrate  feasibility,  we  consider  it  sufficient  to 
define  the  values  of  the  first  three  mode  coefficients.  Together  these  modes  account  for 
more  than  97%  of  the  displacements  of  the  Gulf  Stream.  Higher  accuracy  could  be  attempted 
in  future  work  by  using  more  modes. 

A  second  candidate  data  set  consisted  of  edge  images  produced  using  the  CSED  algorithm. 6 
However,  there  are  only  a  few  dozen  views  of  the  sea  surface  clear  enough  to  span  the 
domain  during  the  18-month  period  spanned  by  the  GOAP  mesoscale  product  data  set. 
Previous  experience  with  neural  network  training  led  us  to  conclude  that  a  substantially 
larger  set  of  images  would  be  required  for  training  to  converge  in  the  presence  of  noise 
(other  SST  fronts)  in  these  images.  Nevertheless,  we  did  obtain  six  warmest-pixel  composite 
SST  images10  and  the  resulting  edge  images,  as  well  as  the  nine  individual  SST  images  that 
contributed  to  the  composites  and  their  edge  images.  In  all  cases  the  edges  were  computed 

using  the  16  x  16  pixel  option  of  the  CSED  software^  because  the  other  options  produced 
perceptibly  more  fine-grained  noise.  Even  though  the  edge  images  could  not  be  used  as  a 
training  set,  we  did  intend  to  use  these  images  as  test  cases  for  the  network's  performance. 

We  chose  the  correlation  between  the  coefficients  produced  by  the  neural  network  and  the 
actual  values  produced  by  the  CEOF  software  from  the  appropriate  GOAP  mesoscale  product 
as  a  measure  of  how  meaningful  the  set  of  CEOF  coefficients  generated  by  the  neural 

network  is.  Specifically,  we  regarded  the  coefficients  in  a  test  case  as  meaningful  it  they 
are  correlated  with  the  actual  values  with  a  correlation  coefficient  of  0.8  or  better.  For 
cases  in  which  there  are  too  few  pairs  to  compute  correlation,  we  considered  agreement  of 
mode  coefficients  within  20%  to  be  meaningful. 

We  used  a  commercial  software  package,  Neuralworks,  to  establish  which  training 
procedure  generates  a  network  whose  output  best  converges  to  the  training  set’s  actual 
coefficients.  "Best,"  in  this  context,  means  fewest  number  of  training  steps  and  the  smallest 
discrepancies  between  the  network  output  and  the  training  set  coefficients.  The  input  in 
this  case  was  the  132  latitude-longitude  pairs  that  define  one  Gulf  Stream,  i.  e.,  264  nodes. 

The  output  is  a  set  of  six  nodes-real  and  imaginary  parts  of  three  CEOF  coefficients.  We  also 
used  the  commercial  software  to  vary  the  number  of  hidden  layers  and  the  number  of  nodes 
in  those  layers.  Parameters  of  the  learning  algorithm,  number  of  nodes,  and  scaling  of 

input  and  output  values  are  all  modified  empirically  at  this  stage  in  order  to  achieve 
convergence. 

The  commercial  software  accesses  only  limited  memory  and  cannot  handle  the  vast  input 

arrays  inherent  in  satellite  imagery.  To  circumvent  these  deficiencies,  the  next  step  was  to 
implement  the  best  training  algorithm  in  C  under  UNIX  (a  virtual  memory  operating 
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system)  so  that  two-dimensional  grid  values  analogous  to  pixel  values  can  be  used  as  input. 
The  increase  in  the  number  of  input  nodes  requires  an  increase  in  the  number  of  hidden 
nodes.  We  emulated  pixel  input  with  a  50  x  50-point  grid,  giving  2,500  input  nodes  -  an 

increase  by  a  factor  of  almost  10  over  the  previous  case. 

At  this  point  it  was  necessary  to  train  this  new  grid-input  network,  rewrite  the  Gulf  Stream 

profiles  into  this  grid-type  format,  and  test  the  grid-input  network  for  convergence.  As 
before,  we  modified  parameters  of  the  learning  algorithm,  number  of  nodes,  and  scaling  of 

input  and  output  values  empirically  at  this  stage  in  order  to  achieve  convergence.  Finally, 

we  trained  this  network  with  only  77  of  the  available  86  GOAP  product  Gulf  Streams,  leaving 
a  randomly-selected  9  as  a  test  set. 

When  convergence  is  achieved,  it  is  appropriate  to  run  the  fully  trained  network  on  any  of 

the  9  test  Gulf  Streams.  The  coefficients  produced  by  the  network  arc  then  compared  to  the 
actual  coefficients  to  determine  whether  they  are  well-correlated. 

We  then  planned  to  test  the  performance  of  the  network  with  an  1R  edge  image  projected 

onto  the  input  grid.  If  the  performance  of  the  network  is  degraded,  it  would  then  be 
necessary  to  define  a  new  training  set  made  up  of  simulated  noisy  edges.  Such  a  set  could  be 
constructed,  c.  g.,  from  GOAP  mcsoscale  products  with  random  sections  of  random  length 
missing,  and  with  random  segments  of  noise  added.  The  statistics  of  this  random  noise  may 
have  to  be  matched  carefully  to  the  statistics  of  SST  front  noise  observed  in  the  six 

composite  images. 

Finally,  we  expected  to  quantify  the  performance  of  the  network  with  statistical  measures 

such  as  mean  and  rms  difference  between  network  and  mcsoscale  product  Gulf  Streams,  and 
significance  level  for  the  differences. 

We  used  Ncuralworks  software  and  several  Intel  80386-based  microcomputers  (386  PC). 
Neuralworks  operates  under  DOS  but  is  limited  in  memory,  so  the  50  x  50  latitude-longitude 
grid  had  to  be  implemented  in  the  virtual-memory  environment  of  UNIX  on  the  386 
machines.  Imagery  is  displayed  on  EGA  graphics  monitors  and  hardcopies  made  on  a 
LaserWriter  for  which  PostScript  code  had  to  be  written. 

5  RESULTS  ACHIEVED 

The  coefficients  generated  with  the  Neuralworks  network  are  in  excellent  agreement  with 

the  actual  cocfficents  in  the  training  set.  This  leads  us  to  conclude  that  a  neural  network 

can  produce  meaningful  coefficients  for  CEOF  modes;  thus,  a  neural  network  can  generate  a 
continuous  Gulf  Stream. 

We  can  quantify  this  agreement  by  computing  for  each  coefficient  a  mean  and  a  variance 

over  the  ensemble  of  86  Gulf  Streams;  these  are  listed  in  Table  1.  Table  1  shows  that  the 
means  of  the  Ncuralworks  network  typically  agree  with  the  actual  mean  within  3.6%.  But 
more  importantly,  the  264-10-6  network  (the  significance  of  this  name  will  be  clear 

shortly)  also  mimics  the  variance  in  Gulf  Stream  coefficients--typical!y  hitting  the 
variance  within  11.8%  also.  The  correlation  coefficient  for  the  six  sets  of  CEOF  mode 
coefficients  is  typically  greater  than  0.98. 
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Table  1.  Mean  and  Variance  of  86  Gulf  Streams  for  1st  3  Modes,  264- 10-6  Network 


RMS  Difference 


Mode 

Actual 

Actual 

264-10-6 

264-10-6 

Between  Network 

Coefficient 

M  e  a  n 

Variance 

Mean 

Variance 

and  Actual 

1 ,  real 

0.36686 

0.01994 

0.32810 

0.01669 

0.00236 

1.  imag. 

0.51797 

0.00703 

0.54308 

0.00362 

0.00526 

2,  real 

0.62294 

0.02015 

0.61439 

0.02122 

0.00041 

2.  imag. 

0.52452 

0.00795 

0.53376 

0.00809 

0.00019 

3,  real 

0.50531 

0.00785 

0.50749 

0.0076! 

0.00009 

3,  imag. 

0.49391 

0.00653 

0.50644 

0.00650 

0.00025 

This  good  agreement  is  obtained  with  the  following  network  parameters  selected  through 

experiment.  The  learning  algorithm  is  back  propagation;'1  convergence  is  achieved  with 
100,000  iterations  through  the  training  set  wiih  the  learning  coefficient  set  to  0.0  and  the 
momentum  coefficient  set  to  0.6.  The  number  of  hidden  layers  is  !;  the  number  of  nodes  on 

that  layer  is  10  (hence  the  network  consists  of  264-10-6  input-hidden-output  nodes, 
respectively).  The  coefficients  placed  or  retrieved  at  the  six  output  nodes  must  be 
separately  sealed  to  range  between  the  values  of  0.2  and  0.8  in  order  for  the  higher  mode 

coefficients  to  converge  closely. 

We  successfully  implemented  a  new  network  in  the  ('  programming  language  that  would 
operate  on  input  nodes  arranged  as  a  5ft  x  50  grid.  The  Mercator  projection  region  between 

34UN  and  45"\  latitude,  and  75° W  and  50"  W  longitude,  is  mapped  onto  this  5n  \  50  g. i  id.  I  lie 
grid's  nominal  resolution  is  50  km.  The  86  GOAP  mesoscale  product  Gulf  Streams  are  mapped 
onto  this  grid  and  used  as  input  to  the  new  nomal  network.  v.T  call  the  new  netwoik  the 

griil-inpul  network,  or  the  2500  40-6  network,  to  distinguish  n  from  the  earlier  network 

This  network  contains  -JO  nodes  in  one  hi 'den  laser,  learns  using  back  propagation,  and 
uses  the  same  scaling  for  the  conversion  between  output  nodes  and  ('POi  eoe!  1 1;  ictfs  as 

b  ’tore.  This  network  is  trained  wph  VI  of  the  86  Gulf  Streams  0  Gull  Streams  .etc.  \ .!  at 

r.rnh  mi  w  me  held  back  for  testing.  Ii  converged  on  its  final  tem  a  Us  a  I  Si  on;!  p.i- 

1  trough  i lie  'laining  se l .  1  his  convergence  took  about  7  das  s  imi.  on  • :  i:  .7  Ml!- 

•n  a  h  t  n  e 
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:iican  within  0.7%  for  the  training  set.  Again,  more  importantly,  the  2500-40-6  network  also 
matches  the  variance  in  Gulf  Stream  coefficients,  typically  coming  within  13.0%  of  the 
correct  variance.  The  correlation  coefficient  for  the  6  sets  of  CFOI  mode  coefficients  is 
greater  than  0.87.  This  is  extremely  encouraging  because  these  good  measures  are  achieved 
without  optimizing  the  network's  parameters.  Of  further  importance  is  that  the  processing 
by  the  network  is  extremely  fast;  it  takes  less  than  1  second  to  produce  three  complex  mode 
coefficients  from  50  x  50  griuded  input. 


Table  3.  Mean  and  Variance  of  9  Test  Gulf  Streams  for  1st  3  Modes.  2500-40-6  Network 

RMS  Difference 


Mode 

Actual 

Actual 

2500-40-6 

2500-40-6 

Rclwcen  Network 

Coefficient 

Mean 

Variance 

Mean 

Variance 

and  Actual 

1.  real 

0.32451 

0.01083 

0.37834 

0.02275 

0.02468 

1,  imag. 

0.53443 

0.00471 

0.52649 

0.00726 

0.00328 

2,  real 

0.65976 

0.01  1  18 

0.62632 

0.02066 

0.01529 

2,  imag. 

0.52343 

0.01317 

0.50572 

0.0045  1 

0.00813 

3,  real 

0.51534 

0.007122 

0.48515 

0.00488 

0.00378 

3,  imag. 

0.46795 

0.01  1  19 

0.48532 

0.00370 

0.01652 

The  grid-input 

network  docs 

less  well  with 

data  in  the  test 

set  (Figure  2 

and  Table  3),  but  still 

achieves  agreement  within  6.0%  for  the  means,  though  only  within  68%  of  the  variances, 
f he  performance  in  matching  the  variance  is  skewed  by  poor  performance  on  just  two  Gulf 
Stream  axes  because  the  test  set  is  so  small. 


The  trained  2500-40-6  network  performs  moderately  well  on  noisy  edge  imagery,  also.  A  512 
x  512  edge  image  is  transformed  to  the  much  coarser  50  x  50  grid  to  provide  a  noisy  input 

data  set.  In  spite  of  looking  nothing  like  the  training  set  or  Gulf  Streams,  the  input  does  not 
cause  the  nc'work  to  diverge.  in  fact,  the  network  generates  quite  reasonable  coefficients. 
The  network  Gulf  Stream  is  within  60  nmi  of  the  actual  Gulf  Stream  for  most  of  its  length, 

and  properly  identifies  Gulf  Stream  SST  edges  by  crossing  over  them.  This  result  is  encour¬ 

aging.  but  is  considered  preliminary  at  this  point. 

fhc  modest  successes  achieved  for  a  network  not  optimized  for  gridded  input  and  not  trained 

lor  noisy  edges  indicate  that  this  neural  network  is  robust,  i.  c.,  not  unstable  over  a  range  of 
parameters.  We  expect  the  network  to  achieve  belter  performance  with  further  experi¬ 

mentation.  The  fast  processing  (less  than  1  second)  of  a  50  x  50  edge  image  by  a  previously 
trained  network  is  a  great  strength  of  the  method. 

6.  PLANS  FOR  NEXT  STAGE  OF  RESEARCH 

I  he  next  logical  step  is  to  extend  the  range  of  the  study  by  expanding  upon  the  number  of 

mode  coefficients  the  network  produces  so  that  a  better,  more  useful  Gulf  Stream 
description  emerges  from  the  system.  The  production  of  more  and  better  mode  coefficients 
requires  increasing  the  resolution  of  the  grid  input  to  the  neural  network.  This,  in  turn, 
requires  a  greater  number  of  computations,  which  suggests  that  special  hardware,  such  as  a 

high-speed  digital  signal  processing  chip,  would  be  advantageous. 

I  he  IR  images  in  which  we  wish  to  locate  the  Gulf  Stream  cover  a  much  larger  area  than 
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that  in  which  the  Gulf  Stream  has  ever  been  located.  The  input  nodes  of  the  network  which 
cover  this  area  contain  no  useful  information  for  the  network,  but  do  increase  the  compu¬ 
tational  burden.  This  suggests  a  logical  first  step  for  future  research.  By  obtaining  the 
envelope  in  which  the  Gulf  Stream  has  been  located  in  the  past  as  defined,  e.  g..  by  GOAP 
mcsoscalc  products,  we  will  be  able  to  eliminate  those  nodes  which  correspond  to  geograph¬ 
ical  areas  through  which  the  Gulf  Stream  has  not  been  known  to  pass.  We  estimate  the 
envelope  will  cover  one-third  of  the  area  now  represented  by  the  input  nodes  to  the  net¬ 
work.  Thus  this  step  would  allow  us  to  increase  the  resolution  of  the  input  grid  by  a  factor 
of  three  without  requiring  any  additional  calculations. 

The  main  impediment  to  increasing  the  resolution  of  the  input  grid  further  (and  therein 
enabling  more  mode  coefficients  to  be  produced)  is  the  large  number  of  computations 
needed  to  train  the  neural  network.  In  order  to  carry  out  the  proposed  steps  for  the  effort 
in  a  reasonable  amount  of  time  additional  processing  power  will  be  required.  As  suggested 
above,  a  (relatively  inexpensive)  processing  board  could  increase  the  speed  of  network 
training  (by  an  estimated  factor  of  50).  This  addition  would  also  require  some  software 
changes,  although  they  would  probably  be  minor. 

Once  the  historical  envelope  of  the  Gulf  Stream  has  been  determined,  the  images  and  Gulf 
Streams  used  in  the  earlier  study  will  need  to  be  regridded  to  cover  the  envelope  so  deter¬ 
mined.  at  a  resolution  increased  by  a  factor  of  3.  Then,  the  previous  neural  network  must 
be  trained  on  the  regridded  data.  This  will  enable  us  to  compare  the  results  to  that  of  the 
previous  work  in  order  to  establish  the  effects  of  increased  resolution  and  the  actual  speed¬ 
up  offered  by  the  additional  processing  power.  Next,  we  will  attempt  to  quantify  the  effects 
of  the  increased  resolution  on  the  quality  of  the  network's  output 

At  that  point  it  will  be  necessary  to  decide  upon  the  more  fruitful  direction  for  further 
research.  We  must  decide  whether  a  higher  resolution  is  needed  to  produce  the  present 
number  of  mode  coefficients  with  sufficient  accuracy,  or  if  more  mode  coefficients  could  be 
produced  at  this  resolution.  Based  on  this  decision,  we  will  modify  the  network  in  one  of  two 
ways.  If  the  results  indicate  that  a  higher  resolution  is  required,  the  images  which  form 
the  training  set  will  be  regridded  at  a  higher  resolution.  If  the  results  indicate  more  mode 
coefficients  could  be  produced  at  the  present  resolution,  then  the  number  of  output  nodes  of 
the  network  will  be  increased. 

It  is  obvious  that  the  cycle  of  testing,  evaluation,  and  modification  could  be  repeated  several 
times.  The  objective  is  to  provide  a  neural  network  system  which  can  automatically  derive  a 
useful  mathematical  (CEOF)  description  of  the  Gulf  Stream  directly  from  edge  images 
produced  from  satellite  observations.  The  criterion  for  usefulness  is  that  the  Gulf  Stream 
description  should  reproduce  that  produced  by  a  skilled  human  analyst  from  the  same  data 
to  a  precision  adequate  to  meet  the  requirements  of  the  U.  S.  Navy.  Such  a  system  has  the 
potential  to  become  a  useful  operational  tool. 
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Figure  1.  Capacity  of  10  Complex  Empirical  Orthogonal  Functions  (CEOFs)  to  reconstrui 
(dashed  line)  even  a  convoluted  Gulf  Stream  produced  by  an  analyst  (solid  line).  The 
analyst  Gulf  Stream  is  from  the  GOAP  mesoscale  product  for  April  18,  1986. 
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Figure  2.  Proximity  of  3  modes  from  grid-input  or  2500-40-6  neural  network  (dashed  line) 
to  the  actual  3  modes  (solid  line)  for  a  test  Gulf  Stream. 
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